You simply need to connect the audio output of your radio to your Mac’s audio input (this is the jack on the back, usually with a little microphone icon). A few models come with audio input cords, most don’t. Also, some models expect an actual microphone to be connected, and require a small adapter (usually included) to allow direct audio to be connected.
Some models require you to select your sound input source, usually from the Sound control panel.
Also, you probably want to tap the audio from your radio’s headphone or external speaker jack, rather than a line level or “Record” output, which may be too low in amplitude. You'll need to experiment. There are too many possible combinations and Mac models for me to give you a definitive answer for your particular setup.
If you want to transmit, you'll need to connect the audio output (speaker) jack of your Mac to your radio's microphone input jack.
For transmitting, you may want to be able to control your radio's PTT (Push To Talk) line. A PTT output is generated from the DTR output of the Mac's serial port. A CW key output is generated from the Transmit Data output. You need to select which port to use (if any) from the Preferences. After selecting the port, re-start MultiMode to make the change effective.
The following schematic shows one way to connect a radio. Please note that dangerous voltages may be present in your radio. It is up to you, the user, to determine how to safely connect to your radio, and prevent any damage to your computer (or yourself!) This type of circuit could also be used in CW mode to key a transmitter.
 
Another (perhaps better) way would be to use an opto-isolator.
As a test, select the serial port connected to your modem. In CW mode, if you type keys, the morse code characters are transmitted out the serial port, causing the TD (or equivilent) light on your modem to flash. Likewise, if you go into transmit mode in RTTY, SSTV, etc modes, the DTR light on the modem will turn on.
